Image processing method, image processing device, and storage medium

ABSTRACT

An image processing method, which is executed by a processor, comprises acquiring choroid information from an image of a fundus of an examined eye, and comparing the choroid information against a choroid normative database and determining whether or not there is an abnormality in the fundus.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on Japanese Patent Application No. 2018-192509filed on Oct. 11, 2018, the disclosure of which is incorporated byreference herein entirely.

BACKGROUND Technical Field

Technology disclosed in the present disclosure relates to an imageprocessing method, an image processing device, and a storage medium.

Related Art

The specification of US Patent Application Laid-Open No. 2015/0366452A1discloses tomographic image analysis of a fundus and extraction of ananomalous region therein. There is a desire to be able to check anabnormality more easily.

SUMMARY

A first aspect of the present disclosure is an image processing method,which is executed by a processor, comprises acquiring choroidinformation from an image of a fundus of an examined eye, and comparingthe choroid information against a choroid normative database anddetermining whether or not there is an abnormality in the fundus.

A second aspect of the present disclosure is an image processing devicecomprising memory and a processor coupled to the memory, wherein theprocessor is configured to acquire choroid information from an image ofa fundus of an examined eye, and compare the choroid information againsta choroid normative database and determine whether or not there is anabnormality in the fundus.

A third aspect of the present disclosure is a storage medium being not atransitory signal and stored with an image processing program thatcauses a computer to execute processing, the processing comprisingacquiring choroid information from an image of a fundus of an examinedeye, and comparing the choroid information against a choroid normativedatabase and determining whether or not there is an abnormality in thefundus.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram of an ophthalmic system 100;

FIG. 2 is a schematic configuration diagram illustrating an overallconfiguration of an ophthalmic device 110;

FIG. 3 is a block diagram of configuration of an electrical system of amanagement server 140;

FIG. 4 is a block diagram illustrating a configuration of an electricalsystem of an image viewer 150;

FIG. 5 is a block diagram illustrating functionality of a CPU 262 of amanagement server 140;

FIG. 6 is a flowchart of an image processing program;

FIG. 7 is a flowchart of UWF-SLO image processing performed at step 212in FIG. 6;

FIG. 8 is a diagram illustrating various types of UWF-SLO imagesobtained by an ophthalmic device 110;

FIG. 9 is a diagram illustrating a choroidal vascular image;

FIG. 10 is a diagram illustrating a choroidal vascular binary image;

FIG. 11 is a diagram illustrating a first region 274 and a second region272 set by a straight line LIN established on a choroidal vascularimage, with analysis points set in each region;

FIG. 12 is a diagram illustrating a region (cell) 244 configured byplural surrounding pixels centered on a central pixel corresponding toan analysis point 242;

FIG. 13 is a diagram illustrating histograms for respective analysispoints;

FIG. 14 is a diagram illustrating a choroidal vascular image 356 inwhich the positions of vortex veins are overlaid on the choroidalvascular image;

FIG. 15 is a diagram illustrating setting of a circle 404 of a specificradius centered on a vortex vein position 402 in a choroidal vascularbinary image;

FIG. 16 is a diagram illustrating intersections 406 between the circle404 and thinned lines;

FIG. 17 is a diagram illustrating a distance image;

FIG. 18 is a diagram illustrating a graph with an angle of positions ofintersections from a specific position on a circle (for example theuppermost edge of the circle) on the horizontal axis, and blood vesseldiameters at the intersections on the vertical axis;

FIG. 19 is a diagram illustrating a histogram of the number of choroidalblood vessels by blood vessel diameter obtained by tallying the bloodvessel diameters at the respective intersections to create a histogramwith six bins, each bin having a width of 200 μm;

FIG. 20 is a flowchart of UWF-OCT image processing performed at step 214in FIG. 6;

FIG. 21 is a diagram illustrating an OCT image (B-SCAN image) 642 of afundus acquired with reference to a straight line and three-dimensionalOCT data 644 for a two-dimensional region acquired with reference to thetwo-dimensional region;

FIG. 22 is a diagram illustrating a choroid 652 obtained from the OCTimage (B-SCAN image) 642, a choroid 654 obtained from thethree-dimensional OCT data 644, and a thickness map 656 of choroidthickness;

FIG. 23 is a diagram illustrating contents stored in a normativedatabase; and

FIG. 24 is a diagram illustrating a display screen 800 for analysisresults displayed on a display 156 of an image viewer 150.

DETAILED DESCRIPTION

Detailed explanation follows regarding exemplary embodiments oftechnology disclosed herein, with reference to the drawings.

Configuration of an ophthalmic system 100 will now be explained withreference to FIG. 1. As illustrated in FIG. 1, the ophthalmic system 100includes an ophthalmic device 110, an eye axial length measurementdevice 120, a management server device (referred to hereafter as“management server”) 140, and an image display device (referred tohereafter as “image viewer”) 150. The ophthalmic device 110 acquires animage of the fundus. The eye axial length measurement device 120measures the axial length of the eye of a patient. The management server140 stores plural fundus images, eye axial lengths, and tomographicimages obtained by imaging the fundus of plural patients using theophthalmic device 110, and stores these in association with patient IDs.The image viewer 150 displays fundus images acquired from the managementserver 140.

The ophthalmic device 110, the eye axial length measurement device 120,the management server 140, and the image viewer 150 are coupled togetherover a network 130.

The eye axial length measurement device 120 has two modes for measuringeye axial length, this being the length of an examined eye 12 in an eyeaxial direction: a first mode and a second mode. In the first mode,after light from a non-illustrated light source is guided into theexamined eye 12, interference between light reflected from the fundusand light reflected from the cornea is photo-detected as interferencelight, and the eye axial length is measured based on an interferencesignal representing the photo-detected interference light. The secondmode is a mode to measure the eye axial length by employingnon-illustrated ultrasound waves.

The eye axial length measurement device 120 transmits the eye axiallength as measured using either the first mode or the second mode to themanagement server 140. The eye axial length may be measured using boththe first mode and the second mode, and in such cases, an average of theeye axial lengths as measured using the two modes is transmitted to themanagement server 140 as the eye axial length.

Explanation follows regarding configuration of the ophthalmic device110, with reference to FIG. 2.

For ease of explanation, scanning laser ophthalmoscope is abbreviated toSLO, and optical coherence tomography is abbreviated to OCT.

In cases in which the ophthalmic device 110 is installed on a horizontalplane with a horizontal direction taken as an X direction, a directionperpendicular to the horizontal plane is denoted as being a Y direction,and a direction connecting the center of the pupil at the anteriorsegment of the examined eye 12 and the center of the eyeball is denotedas being a Z direction. The X direction, the Y direction, and the Zdirection are thus mutually perpendicular directions.

The ophthalmic device 110 includes an imaging device 14 and a controldevice 16. The imaging device 14 is provided with an SLO unit 18 and anOCT unit 20, and acquires a fundus image of the fundus of the examinedeye 12. Two-dimensional fundus images that have been acquired by the SLOunit 18 are referred to hereafter as SLO images. Tomographic images,face-on images (en-face images) and the like of the retina created basedon OCT data acquired by the OCT unit 20 are referred to hereafter as OCTimages.

The control device 16 includes a computer provided with a CentralProcessing Unit (CPU) 16A, Random Access Memory (RAM) 16B, Read-OnlyMemory (ROM) 16C, and an input/output (I/O) port 16D.

The control device 16 is provided with an input-output/display device16E coupled to the CPU 16A through the I/O port 16D. Theinput-output/display device 16E includes a graphical user interface todisplay images of the examined eye 12 and to receive variousinstructions from a user. An example of the graphical user interface isa touch panel display.

The control device 16 is provided with an image processing device 17coupled to the I/O port 16D. The image processing device 17 generatesimages of the examined eye 12 based on data acquired by the imagingdevice 14. Note that the control device 16 is coupled to the network 130through a communication interface, not illustrated in the drawings.

Although the control device 16 of the ophthalmic device 110 is providedwith the input-output/display device 16E as illustrated above in FIG. 2,the technology disclosed herein is not limited thereto. For example, aconfiguration may adopted in which the control device 16 of theophthalmic device 110 is not provided with the input-output/displaydevice 16E, and instead a separate input-output/display device isprovided that is physically independent of the ophthalmic device 110. Insuch cases, the display device is provided with an image processingprocessor unit that operates under the control of a display controlsection 204 of the CPU 16A in the control device 16. Such an imageprocessing processor unit may display SLO images and the like based onan image signal output as an instruction by the display control section204.

The imaging device 14 operates under the control of an imaging controlsection 202 of the control device 16. The imaging device 14 includes theSLO unit 18, an image capture optical system 19, and the OCT unit 20.The image capture optical system 19 includes a first optical scanner 22,a second optical scanner 24, and a wide-angle optical system 30.

The first optical scanner 22 scans light emitted from the SLO unit 18two dimensionally in the X direction and the Y direction. The secondoptical scanner 24 scans light emitted from the OCT unit 20 twodimensionally in the X direction and the Y direction. As long as thefirst optical scanner 22 and the second optical scanner 24 are opticalelements capable of polarizing light beams, they may be configured byany out of, for example, polygon mirrors, mirror galvanometers, or thelike. A combination thereof may also be employed.

The wide-angle optical system 30 includes an objective optical system(not illustrated in FIG. 2) provided with a common optical system 28,and a combining section 26 that combines light from the SLO unit 18 withlight from the OCT unit 20.

The objective optical system of the common optical system 28 may be areflection optical system employing a concave mirror such as anelliptical mirror, a refractive optical system employing a wide-anglelens, or may be a reflection-refractive optical system employing acombination of a concave mirror and a lens. Employing a wide-angleoptical system that utilizes an elliptical mirror, wide-angle lens, orthe like enables imaging to be performed of not only a central portionof the fundus, but also of the retina at the periphery of the fundus.

For a system including an elliptical minor, a configuration may beadopted that utilizes an elliptical mirror system as disclosed inInternational Publication (WO) Nos. 2016/103484 or 2016/103489. Thedisclosures of WO Nos. 2016/103484 or 2016/103489 are incorporated intheir entirety by reference herein.

Observation of the fundus over a wide field of view (FOV) 12A isimplemented by employing the wide-angle optical system 30. The FOV 12Arefers to a range capable of being imaged by the imaging device 14. TheFOV 12A may be expressed as a viewing angle. In the present exemplaryembodiment the viewing angle may be defined in terms of an internalillumination angle and an external illumination angle. The externalillumination angle is the angle of illumination by a light beam shonefrom the ophthalmic device 110 toward the examined eye 12, and is anangle of illumination defined with respect to a pupil 27. The internalillumination angle is the angle of illumination of a light beam shoneonto the fundus F, and is an angle of illumination defined with respectto an eyeball center O. A correspondence relationship exists between theexternal illumination angle and the internal illumination angle. Forexample, an external illumination angle of 120° is equivalent to aninternal illumination angle of approximately 160°. The internalillumination angle in the present exemplary embodiment is 200°.

SLO fundus images obtained by imaging at an imaging angle having aninternal illumination angle of 160° or greater are referred to asUWF-SLO fundus images. UWF is an abbreviation of ultra wide field.

An SLO system is realized by the control device 16, the SLO unit 18, andthe image capture optical system 19 as illustrated in FIG. 2. The SLOsystem is provided with the wide-angle optical system 30, enablingfundus imaging over the wide FOV 12A.

The SLO unit 18 is provided with a blue (B) light source 40, a green (G)light source 42, a red (R) light source 44, an infrared (for examplenear infrared) (IR) light source 46, and optical systems 48, 50, 52, 54,56 to guide the light from the light sources 40, 42, 44, 46 onto asingle optical path using transmission or reflection. The opticalsystems 48, 50, 56 are configured by mirrors, and the optical systems52, 54 are configured by beam splitters. B light is reflected by theoptical system 48, is transmitted through the optical system 50, and isreflected by the optical system 54. G light is reflected by the opticalsystems 50, 54, R light is transmitted through the optical systems 52,54, and IR light is reflected by the optical systems 52, 56. Therespective lights are thereby guided onto a single optical path.

The SLO unit 18 is configured so as to be capable of switching betweenthe light source or the combination of light sources employed foremitting laser light of different wavelengths, such as a mode in which Rlight and G light are emitted, a mode in which infrared light isemitted, etc. Although the example in FIG. 2 includes three lightsources, i.e. the G light source 42, the R light source 44, and the IRlight source 46, the technology disclosed herein is not limited thereto.For example, the SLO unit 18 may, furthermore, also include a blue (B)light source or a white light source, in a configuration in which lightis emitted in various modes, such as a mode in which G light, R light,and B light are emitted or a mode in which white light is emitted alone.

Light introduced to the image capture optical system 19 from the SLOunit 18 is scanned in the X direction and the Y direction by the firstoptical scanner 22. The scanning light passes through the wide-angleoptical system 30 and the pupil 27 and is shone onto the fundus.Reflected light that has been reflected by the fundus passes through thewide-angle optical system 30 and the first optical scanner 22 and isintroduced into the SLO unit 18.

The SLO unit 18 is provided with a beam splitter 64 and a beam splitter58. From out of the light coming from the posterior segment (fundus) ofthe examined eye 12, the B light therein is reflected by the beamsplitter 64 and light other than B light therein is transmitted by thebeam splitter 64. From out of the light transmitted by the beam splitter64, the G light therein is reflected by the beam splitter 58 and lightother than G light therein is transmitted by the beam splitter 58. TheSLO unit 18 is further provided with a beam splitter 60 that, from outof the light transmitted through the beam splitter 58, reflects R lighttherein and transmits light other than R light therein. The SLO unit 18is provided with a beam splitter 62 that reflects IR light from out ofthe light transmitted through the beam splitter 60. The SLO unit 18 isprovided with a B light detector 70 to detect B light reflected by thebeam splitter 64, a G light detector 72 to detect G light reflected bythe beam splitter 58, an R light detector 74 to detect R light reflectedby the beam splitter 60, and an IR light detector 76 to detect IR lightreflected by the beam splitter 62.

Light that has passed through the wide-angle optical system 30 and thefirst optical scanner 22 and been introduced into the SLO unit 18 (i.e.reflected light that has been reflected by the fundus) is reflected bythe beam splitter 64 and photo-detected by the B light detector 70 whenB light, and is reflected by the beam splitter 58 and photo-detected bythe G light detector 72 when G light. When R light, the incident lightis transmitted through the beam splitter 58, reflected by the beamsplitter 60, and photo-detected by the R light detector 74. When IRlight, the incident light is transmitted through the beam splitters 58,60, reflected by the beam splitter 62, and photo-detected by the IRlight detector 76. The image processing device 17 that operates underthe control of the CPU 16A employs signals detected by the B lightdetector 70, the G light detector 72, the R light detector 74, and theIR light detector 76 to generate UWF-SLO images.

As illustrated in FIG. 12, the UWF-SLO images include a UWF-SLO image(green fundus image) 502GG obtained by imaging the fundus in green, anda UWF-SLO image (red fundus image) 504RG obtained by imaging the fundusin red. The UWF-SLO images further include a UWF-SLO image (blue fundusimage) 506BG obtained by imaging the fundus in blue, and a UWF-SLO image(IR fundus image) 508IRG obtained by imaging the fundus in IR.

The control device 16 may control the light sources 40, 42, 44 so as toemit light at the same time as each other. The green fundus image 502GG,the red fundus image 504RG, and the blue fundus image 506BG may beobtained at mutually corresponding each position by imaging the fundusof the examined eye 12 using B light, G light, and R light at the sametime. An RGB color fundus image may be obtained from the green fundusimage 502GG, the red fundus image 504RG, and the blue fundus image506BG. The control device 16 may also control the light sources 42, 44so as to emit light at the same time as each other. The green fundusimage 502GG and the red fundus image 504RG are obtained at mutuallycorresponding positions by imaging the fundus of the examined eye 12using G light and R light at the same time in this manner. An RG colorfundus image may be obtained from the green fundus image 502GG and thered fundus image 504RG.

The UWF-SLO images further include an UWF-SLO image (video) 510ICGGimaged using ICG fluorescent light. When indocyanine green (ICG) isinjected into a blood vessel so as to reach the fundus, the indocyaninegreen (ICG) first reaches the retina, then reaches the choroid, beforepassing through the choroid. The UWF-SLO image (video) 510ICGG is avideo image from the time the indocyanine green (ICG) injected into ablood vessel reached the retina until after the indocyanine green (ICG)has passed through the choroid.

Each image data for the blue fundus image 506BG, the green fundus image502GG, the red fundus image 504RG, the IR fundus image 508IRG, the RGBcolor fundus image, the RG color fundus image, and the UWF-SLO image510ICGG are sent from the ophthalmic device 110 to the management server140 through a non-illustrated communication IF.

An OCT system is realized by the control device 16, the OCT unit 20, andthe image capture optical system 19 illustrated in FIG. 2. The OCTsystem is provided with the wide-angle optical system 30. This enablesfundus imaging to be performed over the wide FOV 12A similarly to whenimaging the SLO fundus images as described above. The OCT unit 20includes a light source 20A, a sensor (detector) 20B, a first lightcoupler 20C, a reference optical system 20D, a collimator lens 20E, anda second light coupler 20F.

Light emitted from the light source 20A is split by the first lightcoupler 20C. After one part of the split light has been collimated bythe collimator lens 20E into parallel light, to serve as measurementlight, the parallel light is introduced into the image capture opticalsystem 19. The measurement light is scanned in the X direction and the Ydirection by the second optical scanner 24. The scanned light is shoneonto the fundus through the wide-angle optical system 30 and the pupil27. Measurement light that has been reflected by the fundus passesthrough the wide-angle optical system 30 and the second optical scanner24 so as to be introduced into the OCT unit 20. The measurement lightthen passes through the collimator lens 20E and the first light coupler20C before being incident to the second light coupler 20F.

The other part of the light emitted from the light source 20A and splitby the first light coupler 20C is introduced into the reference opticalsystem 20D as reference light, and is made incident to the second lightcoupler 20F through the reference optical system 20D.

The respective lights that are incident to the second light coupler 20F,namely the measurement light reflected by the fundus and the referencelight, interfere with each other in the second light coupler 20F so asto generate interference light. The interference light is photo-detectedby the sensor 20B. The image processing device 17 operating under thecontrol of an image processing control section 206 generates OCT images,such as tomographic images and en-face images, based on OCT datadetected by the sensor 20B.

OCT fundus images obtained by imaging at an imaging angle having aninternal illumination angle of 160° or greater are referred to asUWF-OCT images.

Image data of the UWF-OCT images is sent from the ophthalmic device 110to the management server 140 through the non-illustrated communicationIF and stored in a storage device 254.

Note that although in the present exemplary embodiment an example isgiven in which the light source 20A is a swept-source OCT (SS-OCT), thelight source 20A may be from various OCT systems, such as from of aspectral-domain OCT (SD-OCT) or a time-domain OCT (TD-OCT) system.

The eye axial length measurement device 120 in FIG. 1 includes the twomodes for measuring the eye axial length, this being the length of theexamined eye 12 in the eye axial direction (Z direction): the first modeand the second mode. In the first mode, after light from anon-illustrated light source has been guided into the examined eye 12,light arising from interference between the reflected light from thefundus and the reflected light from the cornea is photo-detected, andthe eye axial length is measured based on an interference signal asexpressed by the photo-detected interference light. The second mode is amode in which the eye axial length is measured using ultrasound waves,not illustrated in the drawings. The eye axial length measurement device120 transmits the eye axial length as measured using the first mode orthe second mode to the management server 140. The eye axial length maybe measured using both the first mode and the second mode, and in suchcases an average of the eye axial length as measured using the two modesis transmitted as the eye axial length to the management server 140. Theeye axial length is one type of data about a subject, and the eye axiallength is saved as patient information in the management server 140 aswell as being utilized in fundus image analysis.

Explanation follows regarding a configuration of an electrical system ofthe management server 140, with reference to FIG. 3. As illustrated inFIG. 4, the management server 140 is provided with a computer body 252.The computer body 252 includes a CPU 262, RAM 266, ROM 264, and aninput/output (I/O) port 268. A storage device 254, a display 256, amouse 255M, a keyboard 255K, and a communication interface (I/F) 258 arecoupled to the input/output (I/O) port 268. The storage device 254 is,for example, configured by non-volatile memory. The input/output (I/O)port 268 is coupled to the network 130 through the communicationinterface (I/F) 258. The management server 140 is thus capable ofcommunicating with the ophthalmic device 110, the eye axial lengthmeasurement device 120, and the image viewer 150.

The management server 140 stores each data received from the ophthalmicdevice 110 and the eye axial length measurement device 120 in thestorage device 254.

Explanation follows regarding a configuration of an electrical system ofthe image viewer 150, with reference to FIG. 4. As illustrated in FIG.4, the image viewer 150 is provided with a computer body 152. Thecomputer body 152 includes a CPU 162, RAM 166, ROM 164, and aninput/output (I/O) port 168. A storage device 154, a display 156, amouse 155M, a keyboard 155K, and a communication interface (I/F) 158 arecoupled to the input/output (I/O) port 168. The storage device 154 is,for example, configured by non-volatile memory. The input/output (I/O)port 168 is coupled to the network 130 through the communicationinterface (I/F) 158. The image viewer 150 is thus capable ofcommunicating with the ophthalmic device 110 and the management server140.

Explanation follows regarding various functions implemented by the CPU262 of the management server 140 executing the image processing program.As illustrated in FIG. 5, the image processing program includes adisplay control function, an image processing control function, and aprocessing function. The CPU 262 functions as the display controlsection 204, the image processing control section 206, and a processingsection 208 illustrated in FIG. 6 by the CPU 262 executing the imageprocessing program that includes these functions.

The image processing control section 206 is an example of a “choroidinformation acquisition section” and “determination” of the technologydisclosed herein. As will be described in detail later, choroidinformation includes information relating to a choroidal blood vessel (afeature amount of a choroidal blood vessel), and information relating tothe choroid structure (a feature amount of a choroidal structure).

Detailed explanation follows regarding the image processing performed bythe management server 140, with reference to FIG. 6. The imageprocessing (image processing method) illustrated in the flowchart ofFIG. 6 is realized by the CPU 262 of the management server 140 executingthe image processing program.

The image processing control section 206 executes UWF-SLO imageprocessing at step 212, and executes UWF-OCT image processing at step214. Details regarding the processing of step 212 and step 214 will bedescribed later. At step 216, the processing section 208 saves variousdata obtained by the processing of step 212 and step 214 in the storagedevice 254.

FIG. 7 is a flowchart illustrating the UWF-SLO image processing of step212 in FIG. 6. More specifically, the flowchart includes processing togenerate choroid information by performing image processing on theUWF-SLO images, to compare the generated choroid information against anormative database, and to generate a display screen to displaycomparison results. A user is able to determine whether or not thechoroid is abnormal based on these comparison results.

First, at step 302 the image processing control section 206 acquiresUWF-SLO images from the storage device 254. The UWF-SLO images capturedby the ophthalmic device 110 are stored in the storage device 254.

At step 304, the image processing control section 206 creates achoroidal vascular image (FIG. 9) from the acquired UWF-SLO images inthe following manner. The choroidal vascular image is an image obtainedby performing image processing on the UWF-SLO images to make thechoroidal blood vessels visible. The UWF-SLO images subjected to imageprocessing are, for example, a red fundus image and a green fundusimage.

First, explanation follows regarding information included in the redfundus image and the green fundus image that is required to generate thechoroidal vascular image.

The structure of an eye is one in which a vitreous body is covered byplural layers of differing structure. The plural layers include, fromthe vitreous body at the extreme inside to the outside, the retina, thechoroid, and the sclera. Since red light is of longer wavelength, redlight passes through the retina and reaches the choroid. The red fundusimage 504RG therefore includes information relating to blood vesselspresent within the retina (retinal blood vessels) and informationrelating to blood vessels present within the choroid (choroidal bloodvessels). In contrast thereto, due to green light being of shorterwavelength than red light, green light only reaches as far as theretina. The green fundus image 502GG accordingly only includesinformation relating to the blood vessels present within the retina(retinal blood vessels). This thereby enables a choroidal vascular image(FIG. 9) to be obtained by extracting the retinal blood vessels from thegreen fundus image 502GG and removing the retinal blood vessels from thered fundus image 504RG. The choroidal vascular image is specificallygenerated in the following manner.

The image processing control section 206 extracts the retinal bloodvessels from the green fundus image 502GG by applying black hat filterprocessing to the green fundus image 502GG. Next, the image processingcontrol section 206 removes the retinal blood vessels from the redfundus image 504RG by performing an in-painting processing thereon.Namely, position information for the retinal blood vessels extractedfrom the green fundus image 502GG is employed when performing processingto infill the positions of the retinal blood vessel structure in the redfundus image 504RG using pixel values the same as those of surroundingpixels. The image processing control section 206 then emphasizes thechoroidal blood vessels in the red fundus image 504RG by performingcontrast limited adaptive histogram equalization processing on the imagedata of the red fundus image 504RG from which the retinal blood vesselshave been removed. The choroidal vascular image illustrated in FIG. 9was created in this manner. The created choroidal vascular image isstored in the storage device 254.

Since an eyelid or the like may be included in an image of the choroidalblood vessels, the image processing control section 206 performsprocessing at step 304 to crop the choroidal vascular image to a fundusregion (removing eyelids etc.) so as to generate a choroidal vascularimage (see FIG. 10). The choroidal vascular image (FIG. 9) and thechoroidal vascular image (FIG. 10) are thus images in which thechoroidal blood vessels have been rendered visible that were obtained byperforming image processing on the fundus images.

In the above example, the choroidal vascular image is generated from thered fundus image 504RG and the green fundus image 502GG. However, theimage processing control section 206 may generate a choroidal vascularimage from the green fundus image 502GG and the UWF-SLO image 508IRG.The image processing control section 206 may also generate a choroidalvascular image from the blue fundus image 506BG and one image from outof the red fundus image 504RG or the UWF-SLO image 508IRG.

Furthermore, a choroidal vascular image may also be generated from theUWF-SLO image (video) 510. As described above, the UWF-SLO image (video)510 is a video image from the time indocyanine green (ICG) that has beeninjected into a blood vessel reaches the retina until after theindocyanine green (ICG) has passed through the choroid. The choroidalvascular image may be generated from a video image of a period after theindocyanine green (ICG) has passed through the retina and during whichthe indocyanine green (ICG) is passing through the choroid.

The diameter of blood vessels in the choroid is generally larger thanthe diameter of blood vessels in the retina. Specifically, blood vesselsof a diameter larger than a specific threshold value are choroidal bloodvessels. The choroidal vascular image may accordingly be generated byextracting blood vessels from images in the UWF-SLO image (video) 510taken when the indocyanine green (ICG) is passing through the bloodvessels of the retina and the choroid, and then removing any bloodvessels with a diameter smaller than the specific threshold value.

Plural choroidal blood vessels are included in the choroidal vascularimage. At step 304, the image processing control section 206 furtherextracts the respective choroidal blood vessels from the createdchoroidal vascular image.

At step 306, the image processing control section 206 performs imageprocessing on the choroidal vascular image (FIG. 10) to generate featureamounts for the choroidal blood vessel, this being choroid information.

Note that as described later, the choroidal blood vessel feature amountsinclude (1) a degree of asymmetry in blood vessel flow direction, (2)vortex vein positions and a degree of symmetry in vortex vein positions,(3) histograms and average values of blood vessel diameters at vortexvein vicinity positions, (4) the blood vessel diameters of blood vesselsin the choroidal vascular image and a histogram of the blood vesseldiameters, and (5) the degree of meandering of each of the choroidalblood vessels.

Detailed explanation follows regarding the choroidal blood vesselfeature amounts generated by the image processing performed on thechoroidal vascular image.

(1) Processing to Generate the Degree of Asymmetry in Blood Vessel FlowDirection

First, explanation follows regarding the degree of asymmetry in theblood vessel flow direction. The degree of asymmetry in the blood vesselflow direction is a degree of asymmetry between choroidal blood vesselscorresponding to a first region and a second region obtained by dividingthe choroidal vascular image about a specific straight line.Specifically, the degree of asymmetry in the blood vessel flow directionis found in the following manner.

The image processing control section 206 detects the macula M (see alsoFIG. 12) in the green fundus image 502GG. Specifically, since the maculais a dark region in the green fundus image 502GG, the image processingcontrol section 206 detects as the macula M a region of a specificnumber of pixels having the smallest pixel values in the above greenfundus image 502GG that has been read. A central position of the regionthat includes the darkest pixels is computed as the coordinates of theposition of the macula M, and this is stored in the storage device 254.The image processing control section 206 also detects the position ofthe optic nerve head ONH (see also FIG. 12) in the green fundus image502GG. Specifically, the image processing control section 206 detectsthe position of the optic nerve head in the green fundus image 502GG byperforming pattern matching in the green fundus image 502GG forpredetermined images of optic nerve head, computes the detected positionas the coordinates of the position of the optic nerve head ONH, andstores this position in the storage device 254.

The image processing control section 206 reads the macula M coordinatesand the optic nerve head ONH coordinates. The image processing controlsection 206 sets the respective read coordinates on the choroidalvascular image, and establishes a straight line LIN (see FIG. 11) on thechoroidal vascular image to connect the macula M and the optic nervehead ONH together. Note that since the choroidal vascular image isgenerated from the green fundus image and the red fundus image, thecoordinates of the macula M and the coordinates of the optic nerve headONH as detected in the green fundus image match those of the position ofthe macula M and the position of the optic nerve head ONH in thechoroidal vascular image.

The image processing control section 206 then rotates the choroidalvascular image to make the straight line LIN horizontal.

Analysis points are then set in the following manner. As illustrated inFIG. 11, a first region 274 and a second region 272 of the choroidalvascular image are established by the straight line LIN. Specifically,the first region is positions above the straight line LIN, and thesecond region is positions below the straight line LIN.

The image processing control section 206 then places analysis points240KU at positions in a grid pattern at uniform intervals from eachother in the first region 274, with M (a natural number) rows in theup-down direction and N (a natural number) columns in the left-right(horizontal) direction. In FIG. 11, the number of analysis points in thefirst region 274 is M(3)×N(7)(=L: 21).

The image processing control section 206 then places analysis points240KD in the second region 272, at positions with line symmetry aboutthe straight line LIN to the analysis points 240KU placed in the firstregion 274.

The image processing control section 206 computes the blood vessel flowdirection of the choroidal blood vessels at each of the analysis points.Specifically, the image processing control section 206 repeats thefollowing processing for each of the analysis points. Namely, asillustrated in FIG. 12, for a central pixel corresponding to an analysispoint 242, the image processing control section 206 sets a region (cell)244 configured by plural pixels centered on this central pixel andaround the center image.

The region 244 in FIG. 12 is illustrated after top-to-bottom inversion.This is done to facilitate comparison with a region (cell) 248 thatincludes an analysis point 246 at the upper side of the region 244 andthat forms a pair with the region 244.

The image processing control section 206 then calculates a brightnessgradient direction (expressed as an angle from 0° up to but notincluding 180°, with 0° defined as the direction of the straight lineLIN (horizontal line)) for each pixel in the cell 244, based onbrightness values of the peripheral pixels to the pixel beingcalculated. The gradient direction calculation is performed for all ofthe pixels in the cell 244.

Next, the image processing control section 206 counts the number ofpixels inside the cell 244 that have a gradient direction in each ofnine bins (with each bin width of 20°), these being bins on 0°, 20°,40°, 60°, 80°, 100°, 120°, 140°, and 160° with respect to an anglereference line, in order to create a histogram 242H of the countscorresponding to each of the bins. The angle reference line is thestraight line LIN. The width of a single bin in the histogramcorresponds to 20°. The 0° bin is set with the number (count value) ofpixels in the cell 244 having a gradient direction of from 0° up to butnot including 10°, or a gradient direction of from 170° up to but notincluding 180°. The 20° bin is set with the number (count value) ofpixels in the cell 244 having a gradient direction of from 10° up to butnot including 30°. The count values for the bins 40°, 60°, 80°, 100°,120°, 140°, and 160° are set in a similar manner. Due to there beingnine bins in the histogram 242H, the blood vessel flow direction of theanalysis point 242 is defined as being in one of nine types ofdirection. Note that the resolution of the blood vessel flow directioncan be raised by narrowing the width of each bin and increasing thenumber of bins.

The count values of the respective bins (the vertical axis of thehistogram 242H) are normalized when creating the histogram 242H for theanalysis point 242.

Next, the image processing control section 206 identifies the bloodvessel flow direction of the analysis point from the histogram 242H.Specifically, the angle with the smallest count value is identified asthe blood vessel flow direction of the analysis point 242. Note that thereason the gradient direction with the smallest count is taken as theblood vessel flow direction is the following. Namely, the brightnessgradient is small along the blood vessel flow direction, and thebrightness gradient is large in other directions (for example, there isa large difference between a brightness of a blood vessel and abrightness of something other than a blood vessel). Accordingly, therewill be a small count value for the bin corresponding to the bloodvessel flow direction in a brightness gradient histogram created for allthe respective pixels.

A cell 248 is similarly set for the analysis point 246 and a histogram246H is created therefor. The 160° bin is identified as being the binhaving the smallest count value out of the bins for the histogram 246H.The blood vessel flow direction of the analysis point 246 is thusidentified as being 160°.

The above processing is performed for all of the analysis points in thefirst region and the second region so as to identify the blood vesselflow direction of each of the analysis points set in the choroidalvascular image. Namely, histograms are derived for each of the analysispoint, as illustrated in FIG. 13.

The image processing control section 206 reads the respective upper andlower analysis points (in the first region and the second region), andthe blood vessel flow directions of these points. Specifically, theimage processing control section 206 reads the respective analysispoints and the blood vessel flow direction of these points for eachanalysis point pair having line symmetry about the straight line LIN.

The image processing control section 206 computes a value expressing theasymmetry for each pair of analysis points having line symmetry aboutthe straight line LIN. The value used to express asymmetry is thedifference between the respective blood vessel flow directions therein,with this difference being found using the histograms for each analysispoint in each of the pairs. A difference Δh is found between the countsfor each bin in the respective pair of histograms, and Δh is squared.ΣΔh², i.e. the sum of the Δh² over all bins is then calculated. Largervalues of ΣΔh² mean that the difference between the shapes of thehistograms is greater, and there is accordingly more asymmetry. Smallervalues of ΣΔh² mean the histograms more closely resemble each other, andso the asymmetry is accordingly less.

Note that the value employed to express asymmetry is not limited to thesum of the squared errors in the histograms at the respective pairs ofanalysis points. For example, representative angles may be decided fromthe histograms at the respective pairs of analysis points and anabsolute difference between the representative angles computedtherefrom.

The image processing control section 206 finds an overall average valueof the values expressing the asymmetry at the respective pairs as ablood vessel flow direction degree of asymmetry. The thus found bloodvessel flow direction degree of asymmetry is stored in the RAM 266.

(2) Processing to Generate Vortex Vein Positions and Vortex VeinPosition Symmetry

Explanation follows regarding processing to generate vortex veinpositions. Vortex veins are drains for blood that flowed into thechoroid, with four to six thereof being present on the equator of theeyeball posterior toward the posterior pole of the eyeball.

FIG. 14 illustrates a choroidal vascular image 356 in which thepositions of vortex veins have been overlaid on the choroidal vascularimage. A vortex vein position 376 is encircled in the choroidal vascularimage 356. Three vortex veins are encircled in FIG. 14.

The choroidal blood vessels are extracted from the choroidal vascularimage, and the vortex vein positions are computed based on thedirections of flow of the each choroidal blood vessel. Due to the vortexveins being drains for blood flow, there are plural blood vesselconnected to each of the vortex veins. Namely, the vortex vein positionscan be found by searching the choroidal vascular image for points wherethe flow directions of plural blood vessels converge.

The image processing control section 206 identifies the number of vortexveins, and identifies the coordinates of each of the vortex veins in thechoroidal vascular image as respective vortex vein positions. The imageprocessing control section 206 then stores the number of vortex veinsand the respective vortex vein positions in the RAM 266.

The image processing control section 206 further computes a degree ofsymmetry of the vortex vein positions.

Generally vortex veins are present at positions with line symmetry aboutthe straight line LIN described above. The degree of symmetry of thevortex vein positions may be defined as the amount of displacement of avortex vein position at the lower side of the straight line LIN withrespect to a vortex vein position at the upper side of the straight lineLIN (or conversely, the vortex vein position at the lower side of thestraight line LIN may be employed as the reference). Namely, a pointhaving line symmetry about the straight line LIN to the vortex veinposition at the upper side is found, and then an amount of displacementis found between this point of line symmetry and the actual vortex veinposition at the lower side. The image processing control section 206stores this displacement amount as the degree of symmetry of the vortexvein positions in the RAM 266.

(3) Histogram of Blood Vessel Diameters at Vortex Vein VicinityPositions and Average Value Thereof

Explanation follows regarding a histogram of blood vessel diameters atvortex vein vicinity positions and average value thereof.

The image processing control section 206 binarizes the choroidalvascular image (FIG. 10) by binarizing the pixel value of each pixelbased on a specific threshold value, thereby creating a non-illustratedchoroidal vascular binary image. The choroidal vascular binary image isan image in which the choroidal blood vessels are rendered visible(pixels in regions corresponding to choroidal blood vessels are white,and pixels in regions other than choroidal blood vessels are black). Theimage processing control section 206 extracts images of specific regions(vicinities) in the choroidal vascular binary image that include avortex vein position. The choroidal vascular binary image is created bythe image processing control section 206 binarizing each of the pixelvalues for the pixels in the choroidal vascular image based on thespecific threshold value. This choroidal vascular binary image is animage in which a choroidal blood vessel is visualized (pixels in aregion corresponding to the choroidal blood vessel are white and pixelsin a region other than the choroidal blood vessel are black).

As illustrated in FIG. 15, the image processing control section 206 setsa circle 404 having a specific radius centered on a vortex vein position402 on the choroidal vascular binary image. The specific radius of thecircle 404 is 6 mm. The radius of the circle 404 may be set between 2 mmand 20 mm depending on the analysis required.

The image processing control section 206 performs processing to makelines thinner in the image of the specific region of the choroidalvascular binary image that includes the vortex vein position. Asillustrated in FIG. 16, the image processing control section 206 detectsintersections 406 between the circle 404 and the thinned lines. Asillustrated in FIG. 17, the image processing control section 206generates a distance image from the image of the specific region of thechoroidal vascular binary image that includes the VV position. Thedistance image is an image in which the brightness of lines graduallyincreases on progression from the edges of the lines toward the centerthereof, and, according to the thickness of the lines in the image ofthe specific region of the choroidal vascular binary image that includesthe VV position, the brightness of a central position of the line isbrighter the greater the thickness of the line.

The image processing control section 206 extracts from the distanceimage a brightness value for a position corresponding to eachintersection 406. The image processing control section 206 then convertsthe brightness values of the respective intersections 406 into bloodvessel diameters according to a look-up table stored in advance in thestorage device 254 and expressing correspondence relationships betweenpixel brightness and blood vessel diameter.

As illustrated in FIG. 18, the image processing control section 206creates a graph with an angle of each position of the intersections 406from a specific position on the circle (for example the uppermost edgeof the circle) on the horizontal axis, and the blood vessel diameter atthe respective intersection 406 on the vertical axis. This graph enablesthe thicknesses of blood vessels and the direction the blood vesselextends from the vortex vein position 402 to be visualized.

As illustrated in FIG. 19, the image processing control section 206tallies the blood vessel diameters at the respective intersections 406and creates a histogram of the number of choroidal blood vessels byblood vessel diameter with six bins each bin having a width of 200 μm.This histogram enables a distribution of thicknesses of blood vesselsconnected to the vortex vein to be visualized, thereby enabling a rateof blood flow into the vortex veins etc. to be estimated. Moreover, theimage processing control section 206 computes an average value of theblood vessel diameters for the specific region (vicinity) including thevortex vein position.

The image processing control section 206 stores the created histogram ofthe blood vessel diameters at the vortex vein vicinity positions and theaverage value thereof in the RAM 266.

(4) Computation of Blood Vessel Diameters of Blood Vessels in theChoroidal Vascular Image and Generation of Blood Vessel DiameterHistogram

Explanation follows regarding computation of blood vessel diameters ofblood vessels in the choroidal vascular image and a histogram of bloodvessel diameter. The image processing control section 206 identifiesplural choroidal blood vessels in the choroidal vascular binary imagedescribed above. The image processing control section 206 then computesthe blood vessel diameter for each of the choroidal blood vessels in asimilar manner to in the computation processing for the blood vesseldiameters in the vicinity of the vortex veins as described above. Theimage processing control section 206 then calculates the blood vesseldiameters of the plural choroidal blood vessels.

The image processing control section 206 then generates a blood vesseldiameter histogram from the blood vessel diameters of each of thechoroidal blood vessels.

The image processing control section 206 stores the created blood vesseldiameter histogram and an average value of the blood vessel diameters ofthe blood vessels in choroidal vascular image in the RAM 266.

(5) Degree of Meandering of Choroidal Blood Vessels

Explanation follows regarding the degree of meandering of the respectivechoroidal blood vessels.

The image processing control section 206 make the lines in the choroidalvascular binary image thinner, and finds curves to express therespective choroidal blood vessels. The image processing control section206 then finds the curvature of the curves and the number of meanderingpositions corresponding to each of the choroidal blood vessels.

The image processing control section 206 then stores the curvature ofthe curves and the number of meandering positions in the RAM 266 as adegree of meandering.

As described above, choroidal blood vessel feature amounts are generatedfrom the choroidal vascular image as choroid information. Not only thechoroidal blood vessel feature amounts as described above, but alsovarious other feature amounts of the choroidal blood vessels may becomputed by employing the choroidal vascular images and the UWF-SLOimage.

Next, at step 308, the image processing control section 206 compares therespective choroidal blood vessel feature amounts against a normativedatabase that is stored in a normative database of feature amounts (seeFIG. 23). The image processing control section 206 functions as adecision-making section at step 308. Namely, the image processingcontrol section 206 references a feature amount generated at step 306against a normative database value (DB data) stored in the normativedatabase (see FIG. 23), executes a normal/abnormal determinationfunction, and executes a function to estimate the progression of diseaseand to estimate the prognosis (a predicted time until sight is lost or anumerical value expressing the risk of sight loss).

As illustrated in FIG. 23, the normative database includes databases700A, 700B, . . . for respective conditions as determined by ethnicity,sex, and age. The databases 700A, 700B, . . . are each divided into asection 702 for feature amounts obtained from an SLO image, a section704 for feature amounts obtained from an OCT image, described later, andan indocyanine green angiography (ICGA) video section 706. The section702 for feature amounts obtained from an SLO image includes, for eachfeature amount, a storage region 712 stored with standard values forrespective conditions as determined by ethnicity, sex, and age, and astorage region 714 stored with determination conditions. For example,the storage region 712 of the database 700A for a Japanese male in his20s is stored with standard values for a Japanese male in his 20s. Thestorage region 714 is stored with fixed ranges centered on nominalvalues as normal values.

At step 308, the image processing control section 206 determines whetheror not the respective feature amounts generated at step 306 are normalvalues for each feature amount according to the determination conditionsas stored in the corresponding storage region 714. The determinationresult is sometimes notified using words to express attributes such asnormal or abnormal, and is sometimes a quantitative value such as a riskof sight loss or a predicted time until sight will be lost. Suchquantitative values may be computed by employing a multivariateregression method or the like.

For example, the determination conditions for vortex vein position arenominal values (coordinate values) for each vortex vein position, andnormal ranges set for ranges of a specific distance centered on thesenominal values (coordinate values). The image processing control section206 determines whether or not each of the vortex vein positionsgenerated at step 306 falls within a normal range in the vortex veinposition determination conditions. Abnormal determination is made unlessthe vortex vein position falls within a normal range.

Determination conditions for the degree of symmetry of the vortex veinpositions are normal ranges set for a specific range centered on anominal value of a distance ratio and for a specific range centered on anominal value of an angle ratio. The image processing control section206 determines whether or not the degree of symmetry distance ratio andangle ratio for the vortex vein positions generated at step 306 fall intheir respective normal ranges. Abnormal determination is made when atleast one out of the distance ratio or the angle ratio does not fallwithin the normal range.

Determination conditions for the histogram of diameters of blood vesselsin the vortex vein position vicinity are a normal range set for a rangecentered on a nominal value for the number of choroidal blood vessels ofeach of the blood vessel diameter ranges. Determination conditions forthe average value of the blood vessel diameters in the vortex veinposition vicinity are a normal range set for a range centered on nominalvalues. The image processing control section 206 determines whether ornot the number of choroidal blood vessels in each of the blood vesseldiameter ranges in the histograms generated at step 306 falls in thenormal range, and determines whether or not the blood vessel diameteraverage value generated at step 306 falls in the normal range. The imageprocessing control section 206 makes an abnormal determination in casesin which the number of choroidal blood vessels in any of the bloodvessel diameter ranges does not fall within the normal range, and makesan abnormal determination in cases in which the average value of theblood vessel diameters does not fall within the normal range.

Determination conditions for a histogram of blood vessel diameters inthe whole choroidal vascular image are a normal range set for a rangecentered on a nominal value for the number of choroidal blood vessels ineach of the blood vessel diameter ranges. Determination conditions forthe average value of the blood vessel diameters of the blood vessels inthe choroidal vascular image are a normal range set for a range centeredon a nominal value. The image processing control section 206 determineswhether or not the number of choroidal blood vessels in each of theblood vessel diameter ranges in the histogram generated at step 306 forthe whole choroidal vascular image falls in the normal range, anddetermines whether or not the average value generated at step 306 forthe blood vessel diameter of the blood vessels in the choroidal vascularimage falls in the normal range. The image processing control section206 makes an abnormal determination in cases in which the number ofchoroidal blood vessels in any of the blood vessel diameter ranges doesnot fall in the normal range, and makes an abnormal determination incases in which the blood vessel diameter average value does not fallwithin the normal range.

Determination conditions for the degree of meandering of the respectivechoroidal blood vessels are a normal range set for a range centered on anominal value for a ratio of amplitude to length (cycle) for each of thechoroidal blood vessels. The image processing control section 206determines whether or not the curvature and the number of meanders ofthe choroidal blood vessels generated at step 306 fall within the normalrange. The image processing control section 206 makes an abnormaldetermination in cases in which the curvature and the number of meandersof the choroidal blood vessels do not fall within the normal range.

Determination conditions for the degree of asymmetry in the blood vesselflow direction are a normal range set for a range centered on a nominalvalue for the average value of all the values expressing asymmetry foreach of the analysis point pairs. The image processing control section206 determines whether or not the blood vessel flow direction degree ofasymmetry generated at step 306 falls in the normal range. The imageprocessing control section 206 makes an abnormal determination in casesin which the average value of all the values expressing asymmetry foreach of the analysis point pairs does not fall within the normal range.

Note that a configuration may be adopted in which normal/abnormaldetermination is made on the basis of the interdependency between thefeature amounts instead of being decided according to each of thefeature amounts individually. Namely, although the normal/abnormaldetermination conditions may be set individually for each of the featureamounts, determination may also be made based on multidimensionaldetermination conditions set for multidimensional feature amounts.

Moreover, a configuration may be adopted in which normal/abnormalthreshold value determination basis can be adjusted according to the eyeaxial length of the examined eye.

At step 310, the image processing control section 206 creates displaydata including the comparison results (determination results) determinedat step 308. This display data will be described later.

Detailed explanation follows regarding the UWF-OCT image processingperformed at step 214 in FIG. 6.

FIG. 20 is a flowchart illustrating the UWF-OCT image processingperformed at step 214 in FIG. 6. At step 602, the image processingcontrol section 206 acquires a UWF-OCT image from the storage device254. Obviously an OCT image that is not UWF may also be employed.

A fundus OCT image (B-SCAN image) 642 and three-dimensional OCT data 644(3D-OCT volume data) from a C-SCAN are acquired from the ophthalmicdevice 110 and stored in the storage device 254.

At step 604, the image processing control section 206 performssegmentation of the OCT image 642, identifies each layer thereof, andidentifies the choroid 652 (FIG. 22) based on at least one factor out ofthe number of the layers or a positional relationship to a retinalpigment epithelium (RPE) layer (the layer of greatest pigmentationdensity). Note that the choroid 654 may be three-dimensionallyidentified by performing segmentation of the three-dimensional OCT data644. An OCT angiography (OCT-A) en face image and a choroid thicknessmap 656 may also be created from the three-dimensional OCT data 644. TheOCT-A data may also be employed to analyze the structure of thechoroidal blood vessels.

At step 606, the image processing control section 206 generates featureamounts for the choroid structure, this being choroid informationresulting from performing image processing on the OCT image(three-dimensional OCT data).

Note that the feature amounts for the choroid structure include (1)choroid thickness, (2) the nature of processes at the interface betweenthe choroid and the RPE layer, and the waviness of a curve expressingthis interface, and (3) a lumen/stroma ratio or the like.

(1) Generation of Choroid Thickness Information

The image processing control section 206 identifies the thickness of thechoroid 652 identified in the OCT image 642. A choroid layer 65 isextracted by performing segmentation, and the choroid thicknessinformation is computed by performing image processing thereon. Thecomputed choroid thickness information enables the choroid to berendered visible on the choroid thickness map 656.

The image processing control section 206 stores the choroid thicknessinformation and the choroid thickness map 656 in the RAM 266.

(2) Generation of Nature of Processes and Waviness at Interface BetweenChoroid and RPE Layer

The image processing control section 206 extracts a curve expressing theinterface between the choroid and the RPE layer in the segmented OCTimage. The frequency components of the extracted curve are found byFourier-transformation. The integral (or average value) of highfrequency band components is computed as the waviness. The integral (oraverage value) of low frequency band components is computed as thenature of processes. Note that there is no limitation to performing aFourier transform on the curve, and a wavelet transform may be performedinstead.

Alternatively, the second differential of the curvature of the interfacemay be calculated, and an average value or an integral thereof may becomputed as an index of the nature of processes or the waviness.Alternatively, a difference between, or ratio of, a minimum distance(arc length) between both ends of the interface and a distance (arclength) along the interface may be employed as an index. Alternatively,waves or processes may be counted and the number thereof may be used asan index.

Alternatively, the three-dimensional OCT data 644 may be segmented, theinterface between the choroid and the RPE layer identified, and theabove analysis method may be applied thereto.

Moreover, the nature of processes and the waviness may be computed forthe interface between the choroid and the Bruch's membrane instead forthe interface between the choroid and the RPE layer.

(3) Lumen/Stroma Ratio

The lumen refers to the space inside choroidal blood vessels, andappears black in an OCT image. The stroma refers to portions other thanthe choroid lumen, and appears white in an OCT image. The imageprocessing control section 206 performs segmentation of the OCT imageand identifies a choroid region. Pixels in a brightness range of thelumen region are then counted, and the area of the lumen region is thencomputed by adding up the area for a single pixel over the number ofpixels counted. Similarly, pixels in a brightness range of the stromaregion are counted, and the area of the stroma region is then computedby adding up the area for a single pixel over the number of pixelscounted.

The image processing control section 206 finds the lumen/stroma ratiofrom the area of the lumen region and the area of the stroma region. Thecomputed ratio is then stored in the RAM 266.

As described above, feature amounts for the choroid structure aregenerated from the OCT image and three-dimensional OCT data as choroidinformation. In addition to the choroid structure feature amountsdescribed above, various other choroidal blood vessel feature amountsmay be computed by employing data other than the OCT image.

At the next step 608, the image processing control section 206 comparesthe respective feature amounts obtained from the OCT image against therelevant normative database values stored in the normative database offeature amounts (see FIG. 23). At step 608, the image processing controlsection 206 functions as a decision-making section. Namely, the imageprocessing control section 206 references the feature amounts generatedat step 606 against the relevant normative database values (DB data)stored in the normative database (see FIG. 23) and executes a functionto perform normal/abnormal determination, and to estimate theprogression of disease and the prognosis (a predicted time until sightis lost or a numerical value expressing the risk of sight loss).

As illustrated in FIG. 23, the respective databases 700A, 700B, . . .each include a section 704 for feature amounts obtained from OCT images.The section 704 for feature amounts obtained from OCT images includes,for each feature amount, the storage region 712 stored with standardvalues for respective conditions as determined by ethnicity, sex, andage, and the storage region 714 stored with the determinationconditions. For example, the storage region 712 of the database 700A fora Japanese male in his 20s is stored with standard values for a Japanesemale in his 20s. The storage region 714 is stored with fixed rangescentered on nominal values as normal values.

At step 608, the image processing control section 206 determines whetheror not the respective feature amounts generated at step 606 are normalvalues for the respective feature amounts according to the determinationconditions stored in the corresponding storage region 714.

For example, determination conditions for the choroid thickness arenominal values for choroid thickness and normal ranges set as specificranges centered on the nominal values. The image processing controlsection 206 determines whether or not the choroid thickness generated atstep 606 falls in the normal range for the choroid thicknessdetermination conditions. The image processing control section 206 makesan abnormal determination in cases in which the choroid thickness doesnot fall within the normal range.

The determination conditions for the nature of processes degree ofprojection at the interface between the choroid and the RPE layer are anominal value for the nature of processes and a normal range set as aspecific range centered on the nominal value. The image processingcontrol section 206 determines whether or not the nature of processesgenerated at step 606 falls in the normal range for the nature ofprocesses determination conditions. The image processing control section206 makes an abnormal determination in cases in which the nature ofprocesses does not fall within the normal range.

Determination conditions for the degree of the waviness of the interfacebetween the choroid and the RPE layer are a nominal value for thewaviness and a normal range set as a specific range centered on thenominal value. The image processing control section 206 determineswhether or not the waviness generated at step 606 falls in the normalrange according to the waviness determination conditions. The imageprocessing control section 206 makes an abnormal determination in casesin which the waviness does not fall within the normal range.

Determination conditions for the lumen/stroma ratio are a nominal valuefor the lumen/stroma ratio and a normal range set as a specific rangecentered on the nominal value. The image processing control section 206determines whether or not the lumen/stroma ratio generated at step 606falls in the normal range according to the lumen/stroma ratiodetermination conditions. The image processing control section 206 makesan abnormal determination in cases in which the lumen/stroma ratio doesnot fall within the normal range.

Note that a configuration may be adopted in which normal/abnormaldetermination is made on the basis of the interdependency between thefeature amounts instead of being decided according to each of thefeature amounts individually. Namely, although the normal/abnormaldetermination conditions may be set individually for each of the featureamounts, determination may also be made based on multidimensionaldetermination conditions set for multidimensional feature amounts.

Moreover, a configuration may be adopted in which normal/abnormalthreshold value determination standards can be adjusted according to theeye axial length of the examined eye.

At step 610, the image processing control section 206 creates displaydata including comparison results (determination results) as determinedat step 308. This display data will be described later.

Processing returns to step 216 of FIG. 6 when the processing of step 610has been finished, and the image processing control section 206 storesthe created feature amount data and a display screen in the RAM 266 orthe storage device 254. The processing of FIG. 6 is then ended.

Explanation follows regarding a display screen displaying the results ofcomparing the feature amounts for the examined eye image against thenormative database.

Based on user instruction, the display control section 204 creates adisplay screen 800 (see FIG. 24) to display comparison results from thedisplay data saved at step 216 in FIG. 6 (the results of comparisonagainst the normative database) and an image captured by the ophthalmicdevice 110. The processing section 208 then transmits display screendata for the display screen 800 to the image viewer 150.

On receipt of the display screen data, the image viewer 150 displays thedisplay screen 800 illustrated in FIG. 24 on the display 156 based onthe display screen data.

Explanation follows regarding the display screen 800 illustrated in FIG.24. As illustrated in FIG. 24, the display screen 800 includes apersonal information display area 802 to display personal informationabout a patient, an image display area 804, a choroid analysis tooldisplay area 806, and a finish button 808.

The personal information display area 802 includes a patient ID displayfield 812, a patient name display field 814, an age display field 816, aright eye/left eye display field 818, an eye axial length display field820, a visual acuity display field 822, and an imaging date/time displayfield 824. Each of various types of information is respectivelydisplayed in the display fields 812 to 824. Note that a patient list isdisplayed on the display 156 of the image viewer 150 when an illustratedpatient selection icon is clicked to prompt the user to select thepatient for analysis.

The choroid analysis tool display area 806 is an area to display variousicons to select plural choroid analysis. A vortex vein position icon852, a symmetry icon 854, a blood vessel diameter icon 856, a vortexvein-macula/optic nerve head icon 858, and a choroid analysis reporticon 860 are provided in the choroid analysis tool display area 806. Thevortex vein position icon 852 is used to instruct display of the vortexvein positions. The symmetry icon 854 is used to instruct display of thesymmetry analysis points. The blood vessel diameter icon 856 is used toinstruct display of analysis results relating to the diameter ofchoroidal blood vessels. The vortex vein-macula/optic nerve head icon858 is used to instruct display of analysis results of analyzedpositions between the vortex veins, the macula, and the optic nervehead. The choroid analysis report icon 860 is used to instruct displayof a choroid analysis report.

The image display area 804 performs first various types of display basedon respective comparison results of the feature amounts for thechoroidal blood vessels in the choroid information against the normativedatabase values stored in the feature amount normative databases. Theimage display area 804 performs various types of display based on secondeach comparison results of the respective feature amounts for thechoroid structure in the choroid information against the normativedatabase values stored in the feature amount normative database.

The image display area 804 is a display area to display a choroidanalysis report when the choroid analysis report icon 860 is operated.The image display area 804 includes a choroidal vascular image displayfield 832, a vascular image display field 834, an all-choroidal vascularimage analysis result display field 838AB, and a vortex vein analysisresult display field 840AB. The all-choroidal vascular image analysisresult display field 838AB includes feature amount item display fields838A1 to 838A3, and analysis result display fields 838B1 to 838B3corresponding to each item. The vortex vein analysis result displayfield 840AB includes feature amount item display fields 840A1 to 840A3,and analysis result display fields 840B1 to 840B3 corresponding to eachitem.

A choroidal vascular image is displayed in the choroidal vascular imagedisplay field 832. A vascular image is displayed in the vascular imagedisplay field 834. The numbers in the vascular image display field 834are vortex vein discrimination numbers. The blood vessel diameter ofonly vortex vein #1 is large. The symmetry is also poor on the ear side(#1 and #2 vortex veins).

Regarding the comparison results of step 308 in FIG. 7, consider a casein which there is no abnormality in the blood vessel diameter histogramsnor in the average value blood vessel diameter, but the blood vesselsymmetry is abnormal on the ear side. In such a case, “NO ABNORMALITY”would be displayed in the analysis result display field 838B1corresponding to the blood vessel diameter histogram item display field838A1. Moreover, “NO ABNORMALITY” would be displayed in the analysisresult display field 838B2 corresponding to the blood vessel diameteraverage value feature amount item display field 838A2. However, “EARSIDE: REQUIRES EXAMINATION” would be displayed in the analysis resultdisplay field 838B3 corresponding to the blood vessel symmetry itemdisplay field 838A3.

Regarding the comparison results of step 308 in FIG. 7, consider a casein which there is no abnormality in the vortex vein positions, but theblood vessel diameter histogram for vortex vein #1 is abnormal and theblood vessel diameter average value for vortex vein #1 is abnormal. Insuch a case, “NO ABNORMALITY” would be displayed in the analysis resultdisplay field 840B1 corresponding to the vortex vein position itemdisplay field 840A1. However, “#1: REQUIRES EXAMINATION” would bedisplayed in the analysis result display field 840B2 corresponding tothe blood vessel diameter histogram item display field 840A2. Moreover,“#1: REQUIRES EXAMINATION” would also be displayed in the analysisresult display field 840B3 corresponding to the blood vessel diameteraverage value item display field 840A3.

The various types of first display and the various types of seconddisplay in the image display area 804 are not limited to the examplesillustrated in FIG. 24, and display may be performed based on comparisonresults for various other feature amounts.

Display may be performed based on the comparison results for whether ornot the distance ratio and the angle ratio of the degree of symmetry ofthe vortex vein positions fall in their respective normal ranges, ordisplay may be performed based on the comparison results for whether ornot the curvature and number of meanders of the choroidal blood vesselsfall in the normal range.

Display may also be performed based on the comparison results forwhether or not the choroid thickness falls in the normal range accordingto the choroid thickness determination conditions.

Display may also be performed based on the comparison results forwhether or not the nature of processes at the interface between thechoroid and the RPE layer falls in the normal range according to thenature of processes determination conditions.

Display may also be performed based on the comparison results forwhether or not the waviness at the interface between the choroid and theRPE layer falls in the normal range according to the wavinessdetermination conditions.

Display may also be performed based on the comparison results forwhether or not the lumen/stroma ratio falls in the normal rangeaccording to the lumen/stroma ratio determination conditions.

Display may also be performed based on the results of normal/abnormaldetermination taking into consideration the interdependency between thefeature amounts.

Display based on the respective comparison results may also be performedby using words to express attributes such as normal or abnormal, ordisplay may be performed by using quantitative values such as a risk ofsight loss or predicted time until sight is lost.

Explanation follows regarding various modified examples of thetechnology disclosed herein.

FIRST MODIFIED EXAMPLE

As described above, the normative database (see FIG. 23) includes thedatabases 700A, 700B, . . . for respective conditions as determined byethnicity, sex, and age. The databases 700A, 700B, . . . each include anICGA video section 706. The ICGA video section 706 includes a pulse ofpulsatile polyps and a choroidal capillary filling delay as featureamounts. The storage region 712 is stored with standard values, and thestorage region 714 is stored with determination conditions.

The normative database may be further stored with analysis results ofthe standard values for conditions as determined by ethnicity, sex, andage. For example, the mean, variance, kurtosis, skewness, and alsohigher-order moments of the respective feature amounts, as well ascomposite feature amount data for the respective feature amounts, andthe change over time of the respective feature amounts, may also bestored therein.

Determination conditions are not limited to storing normal ranges, andconditions such as the following may be stored. Stored nominal valuesmay include 3σ, 6σ, or the like computed with reference to the standarddeviation σ of the population of the normative database, the Fischer'sdiscriminant and Z-factor. A risk percentage conversion equation, aprognosis (time until symptoms appear) conversion equation, ordetermination attribute data (abnormal/healthy/high risk) may also bestored.

The contents of the normative database may be updated at specificintervals.

SECOND MODIFIED EXAMPLE

Although in the exemplary embodiment and the first modified exampledescribed above the feature amounts were analyzed according todetermination conditions, the technology disclosed herein is not limitedthereto. For example, changes in historical data of previous featureamounts for a given patient may be analyzed according to determinationconditions for such changes.

THIRD MODIFIED EXAMPLE

Although in the exemplary embodiments described above the managementserver 140 executes the image processing program illustrated in FIG. 6,the technology disclosed herein is not limited thereto. A configurationmay be adopted in which the image viewer 150 transmits an imageprocessing command to the management server 140, with the managementserver 140 executing the image processing program of FIG. 6 in responseto this command.

FOURTH MODIFIED EXAMPLE

In the exemplary embodiments described above, explanation has been givenregarding examples in which a fundus image having an internalillumination angle of approximately 200° is acquired by the ophthalmicdevice 110. The technology disclosed herein is not limited thereto, andthe technology disclosed herein may also be applied in a configurationin which a fundus image having an internal illumination angle of 100° orless is captured by an ophthalmic device, or in a configuration in whicha montage image synthesized from plural fundus images is employed.

FIFTH MODIFIED EXAMPLE

Although in the exemplary embodiments described above the fundus imageemployed is captured by the ophthalmic device 110 provided with an SLOimaging unit, the technology disclosed herein may also be applied to aconfiguration in which a fundus image captured by a fundus cameracapable of imaging choroidal blood vessels, or to an image obtained byOCT angiography, is employed.

SIXTH MODIFIED EXAMPLE

Although in the exemplary embodiments described above asymmetry in thechoroidal blood vessel flow direction is analyzed, the technologydisclosed herein may be applied in analysis of asymmetry in retinalblood vessel flow direction.

SEVENTH MODIFIED EXAMPLE

Although in the exemplary embodiments described above the managementserver 140 executes the image processing program, the technologydisclosed herein is not limited thereto. For example, the ophthalmicdevice 110 or the image viewer 150 may execute the image processingprogram.

EIGHTH MODIFIED EXAMPLE

Although explanation has been given in the exemplary embodimentsdescribed above regarding examples in which the ophthalmic system 100 isprovided with the ophthalmic device 110, the eye axial lengthmeasurement device 120, the management server 140, and the image viewer150, the technology disclosed herein is not limited thereto. Forexample, as a first example, a configuration may be adopted in which theeye axial length measurement device 120 is omitted and the ophthalmicdevice 110 further includes the functionality of the eye axial lengthmeasurement device 120. Alternatively, as a second example, aconfiguration may be adopted in which the ophthalmic device 110 furtherincludes the functionality of at least one out of the management server140 or the image viewer 150. For example, the management server 140 maybe omitted in cases in which the ophthalmic device 110 includes thefunctionality of the management server 140. In such cases, the imageprocessing program is executed by the ophthalmic device 110 or the imageviewer 150. Alternatively, the image viewer 150 may be omitted in casesin which the ophthalmic device 110 includes the functionality of theimage viewer 150. As a third example, a configuration may be adopted inwhich the management server 140 is omitted, and the image viewer 150executes the functionality of the management server 140.

OTHER MODIFIED EXAMPLES

The data processing as explained in the exemplary embodiments describedabove are merely examples thereof. Obviously, unnecessary steps may beomitted, new steps may be added, or the processing sequence may berearranged within a range not departing from the spirit of the presentdisclosure.

Although explanation has been given in the exemplary embodimentsdescribed above regarding an example in which a computer is employed toimplement data processing using a software configuration, the technologydisclosed herein is not limited thereto. For example, instead of asoftware configuration employing a computer, the data processing may beexecuted solely by a hardware configuration such as a field programmablegate array (FPGA) or an application specific integrated circuit (ASIC).Alternatively, a configuration may be adopted in which some processingout of the data processing is executed by a software configuration, andthe remaining processing is executed by a hardware configuration.

What is claimed is:
 1. An image processing method executed by aprocessor, the image processing method comprising: acquiring choroidinformation from an image of a fundus of an examined eye; and comparingthe choroid information against a choroid normative database anddetermining whether or not there is an abnormality in the fundus,wherein the choroid information includes a feature amount relating tochoroid structure.
 2. The image processing method of claim 1, furthercomprising outputting a determination result from the determination. 3.The image processing method of claim 1, further comprising outputting adetermination result from the determination and outputting the fundusimage.
 4. The image processing method of claim 1, wherein the fundusimage is an SLO image acquired by a scanning laser ophthalmoscope (SLO).5. The image processing method of claim 1, wherein the fundus image isan OCT image acquired by optical coherence tomography (OCT).
 6. Theimage processing method of claim 1, wherein the choroid informationincludes a feature amount relating to a choroidal blood vessel.
 7. Theimage processing method of claim 1, wherein the normative databaseincludes at least one standard data out of standard data for a featureamount relating to choroid structure or standard data for a featureamount relating to a choroidal blood vessel.
 8. The image processingmethod of claim 6, wherein: the choroid information includes thechoroidal blood vessel feature amount; and the choroidal blood vesselfeature amount includes a feature amount relating to a vortex vein. 9.The image processing method of claim 7, wherein: the choroid structurefeature amount includes a feature amount relating to choroid thickness.10. An image processing device comprising memory and a processor coupledto the memory, wherein: the processor is configured to acquire choroidinformation from an image of a fundus of an examined eye; and comparethe choroid information against a choroid normative database anddetermine whether or not there is an abnormality in the fundus, whereinthe choroid information includes a feature amount relating to choroidstructure.
 11. A storage medium being not a transitory signal and storedwith an image processing program that causes a computer to executeprocessing, the processing comprising: acquiring choroid informationfrom an image of a fundus of an examined eye; and comparing the choroidinformation against a choroid normative database and determining whetheror not there is an abnormality in the fundus, wherein the choroidinformation includes a feature amount relating to choroid structure.